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METHOD AND APPARATUS FOR TWO-PORT ALLPASS COMPENSATION 
OF POLARIZATION MODE DISPERSION 

Field of the Invention 

5 The present invention relates generally to optical fiber transmission 

systems and, more particularly, to methods and apparatus for measuring and controlling 
polarization mode dispersion (PMD) in such optical fiber transmission systems. 



Background of the Invention 

10 Optical communication systems increasingly employ wavelength division 

multiplexing (WDM) techniques to transmit multiple information signals on the same 
fiber at different optical frequencies. WDM techniques are being used to meet the 
increasing demands for improved speed and bandwidth in optical transmission 
applications, including fiber optic communication systems. 

15 Polarization mode dispersion (PMD) is a fundamental problem in single- 

mode fiber optic communication systems that has limited the channel capacity. Fiber 
asymmetry gives rise to random birefringence effects, resulting in differential group delay 
(DGD) between the two principle polarization modes. Wavelength dependent coupling of 
the modes then leads to polarization mode dispersion, which becomes the dominant 

20 limiting factor of transmission rate as bandwidth increases. 

A need therefore exists for improved techniques for compensating for 
polarization mode dispersion in order to further improve the efficiency and channel 
capacity of high-speed fiber optic communication. 



25 Summary of the Invention 

Generally, a method and apparatus are disclosed for compensating for 
polarization mode dispersion using cascades of all-pass filters and directional couplers. 
The disclosed PMD compensator adjusts the coefficients of an adaptive filter structure 
involving all-pass filters and directional couplers based on a minimized cost function. In 
30 one implementation, a stochastic gradient algorithm, also referred to as the least mean 
square (LMS) algorithm, is employed to sequentially reduce the value of the cost function 
by the method of steepest descent. In another implementation, convergence is improved 
by employing a Newton algorithm that uses second derivatives to accelerate convergence. 
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A more complete understanding of the present invention, as well as further 
features and advantages of the present invention, will be obtained by reference to the 
following detailed description and drawings. 

5 Brief Description of the Drawings 

FIG. 1 is a schematic block diagram of a conventional optical receiver that 
employs a PMD compensator; 

FIG. 2 is a schematic block diagram illustrating of a conventional optical 
fiber transmission system that employs a PMD compensator; 
10 FIG. 3 is a schematic block diagram of the PMD measuring apparatus of 

FIG. 1; 

FIG. 4 is a schematic block diagram of a polarization mode dispersion 
compensator for use in an optical receiver; 

FIG. 5 illustrates an exemplary model of a PMD generator; and 
15 FIG. 6 illustrates an exemplary model of a PMD compensator. 

Detailed Description 

The present invention provides a method and apparatus for compensating 
for polarization mode dispersion using cascades of all-pass filters and directional couplers. 

20 The disclosed PMD compensator adjusts the coefficients of an adaptive filter structure 
involving all-pass filters and directional couplers based on a minimized cost function. 
Initially, the phase and amplitude of polarization components are evaluated in order to 
characterize the PMD of channels and adjust tunable PMD compensators. FIG. 1 is a 
schematic block diagram of a conventional optical receiver 100 that employs a PMD 

25 compensator 400, discussed below in conjunction with FIG. 4. Generally, the PMD 
compensator 400 employs a PMD measuring apparatus 300, discussed further below in 
conjunction with FIG. 3, to measure the PMD and an adaptive optical filter 110 to 
compensate for the measured PMD. The PMD compensator 400 may be integrated with 
or in the vicinity of the optical receiver 100. 

30 As shown in FIG. 1, an information signal is received by the optical 

receiver 100 over an optical fiber 105. The PMD compensator 400 measures the phase 
and magnitude of the polarization components and generates a feedback signal for 
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controlling one or more adaptive optical filters 300 to reduce polarization mode 
dispersion. 

FIG. 2 is a schematic block diagram illustrating of a conventional optical 
fiber transmission system 200 that employs the PMD compensator 400. A test signal 222 
5 is added to a data signal 221 in the form of a comb of tones with known relative 
magnitude and phase relationships. The tones can be equally spaced. The spacing 8 f 
between the tones may be on the order of 2.5 to 5 GHz. The tones provide test signal 
information with a higher signal-to-noise ratio at the receiver than a pseudo-random bit 
sequence. There are many ways to add such a test signal 222, including modifying the 

10 data format such that tones with the desired frequency spacing appear in the output 
spectrum 223. The combined data signal and test signal is fed to a laser/modulator 
combination 224 to generate a wavelength channel (as a channel in a wavelength division 
multiplexed (WDM) transmission system). The channel is combined with other channels 
in an optical multiplexer 225 to form a composite light signal. The composite light signal 

1 5 is then transmitted via optical transmission path 213. 

At a receiver, demultiplexer 226, preferably at the downstream end of 
optical transmission path 213, separates the wavelength channels. Optical coupler 227 
provides a wavelength channel to the polarization mode dispersion compensator (PMDC) 
1 10 (the adaptive optical filter). Coupler 227 also provides light to the PMD measuring 

20 apparatus 300. The channel estimate made at PMD measuring apparatus 300 generates an 
adaptive PMD correction signal to control PMDC 110. Receiver (RX) 100 receives the 
compensated wavelength channel signal. The RF detectors (not shown in FIG. 2) of the 
channel estimate signal analyzer 300 need only have enough bandwidth to accommodate 
5 f and not the whole signal bandwidth. The problem is to determine the relative phase 

25 between each pair of tones when they are all present at the detector. 

FIG. 3 schematically illustrates an exemplary PMD measuring apparatus 

300 for analyzing the test signal downstream. Each polarization is split into a separate 
path by a polarization beam splitter (PBS) and polarization controller (PC) 301. The PC 

301 allows the power in the x- and y-outputs to be controlled so that all of the power is not 
30 in one output or the other. One polarization of the output of the PBS/PC is flipped by a 90 

degree rotation 302 so that the outputs going into the 3 dB couplers 303 have the same 
polarization, i.e., either TE or TM. After the 3 dB coupler, one portion of each 
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polarization is analyzed by a tunable narrowband filter (NBF) 304 to obtain the magnitude 
across the channel (detectors X 308, Y 309), and <f> the relative phase between 
polarization (detector 310) derived via 3 dB coupler 303, while the other portion is 
transmitted through a tunable all-pass filter (APF) 305 before being detected. 
5 Both polarizations see the same narrowband filter but in 

counterpropagating directions. The filters are tuned, for example, by thermo-optic phase 
shifters. The APF's are identical in principle, but any variations can be handled by 
calibration. Each APF is designed to provide a very sharp transition in its phase response 
from 0 to 2 n near resonance. On-resonance, the phase is n . Off-resonance, the phase is 

10 ideally 0 or 2n . As the resonant frequency is shifted via a phase shifter in the feedback 
path, the phase response is translated across the channel spectrum and the RF detectors X 
306 and Y 307 record different linear combinations of beats between adjacent tones. An 
RF reference 313 is obtained from a light signal tapped before the polarization beam 
splitter 301 and fed to an RF detector and phase locked loop incorporating a voltage 

15 controlled oscillator 314 to develop an RF reference signal 313. Blocks 311 and 312 
measure the phase of the X and Y components as detected by 306 and 307 with respect to 
the RF reference 313. For a detailed discussion of the PMD measuring apparatus 300, see 
United States Patent Application Serial Number 10/180,842, entitled "Apparatus and 
Method for Measurement and Adaptive Control of Polarization Mode Dispersion in 

20 Optical Fiber Transmission Systems," incorporated by reference herein. 

According to another aspect of the invention, adaptive algorithms are 
provided for two-channel (two-input/two-output) structures consisting of multiple 
cascades of all-pass filters and directional couplers. While this exemplary architecture is 
employed for compensation of polarization mode dispersion, the results apply more 

25 generally to this class of filters, as would be apparent to a person of ordinary skill in the 
art. For a detailed discussion of an application of this architecture to PMD compensation 
using multiple cascades of all-pass filters and directional couplers, see C.K. Madsen, 
"Optical All-Pass Filters for Polarization Mode Dispersion Compensation," Optics Letters, 
Vol. 25, No. 12, 878-80 (June, 2000), incorporated by reference herein. 

30 FIG. 4 is a schematic block diagram of a conventional polarization mode 

dispersion compensator 400 for use in an optical receiver, such as the optical receiver 100 
of FIG. 1. As shown in FIG. 4, the PMD compensator 400 is a two-channel (two- 
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input/two-output) structure consisting of multiple cascades of all-pass filters and 
directional couplers. In particular, the PMD compensator 400 includes a polarization 
beam splitter 410, two pair of multistage all-pass filters 420-1, 420-2, 440-1, 440-2 and 
two directional couplers 430, 450, that all operate in a known manner. 

Models for PMD Generation and Compensation 
FIG. 5 illustrates an exemplary model of a PMD generator, where the two 
dimensional polarization z-transform vector x(z) generated by the transmitter produces 

the fiber output polarization vector y(z)at the receiver. A opt 540 and B opt 520 are 

multistage all-pass filters and 

V5M -A (1) 



10 T = 

2 {-j 1 

is a unitary matrix representing a directional coupler 510, 530 with parameter value 0.5 
and / = In this manner, the simplified PMD compensator 600 of FIG. 6 has the 

ideal capability of complete compensation, i.e., the compensator output x{z) = x(z) , when 
A = A opt and B = B opt . This will allow observations about the behavior of parameter 

15 estimation algorithms to be made without being overly burdened by the complications of 
real PMD. 

The multistage all-pass filter A 540 is mathematically expressed as: 
A(z) = flA p (z), (2) 

where P is the number of stages and the response of each stage is written as follows: 

z~ x -a 

20 ^W = - 5-, (3) 

l-a p z 

where a p is a complex number that specifies the pole location and * denotes complex 
conjugate. (The complementary zero location is at 1/ a p .) The sections A p {z) are all- 
pass functions so that |^(^)| = 1 at all frequencies. 

Likewise, a Q-stage filter B 520 is defined as 

25 B(z) = f[B q {z\ (4) 

9 =1 



y= 2 



(6) 



. 1 

x = — 
2 



(7) 
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where Q is the number of stages and the response of each stage is written as follows: 

BAz) = V (5) 

qK ' \-b q z- x 

Cascading the above in the order of FIG. 5 and multiplying out the 2x2 
matrices gives the overall response of the PMD model: 

'< Pl (B: Pi -i)jA: Pt (B' opt +\) 

j(B opl+ l) -(5 0 * pt -l) _ 

This shows how the input polarization signals are coupled to the output 
signals, as determined by the all-pass model filters A opt 540 and B opt 520. When 

Apt = ^o P t = ~hy = x and there is no distortion. 

Now consider the response of the PMD compensator 600 of FIG. 6, which 
10 is similarly calculated as: 

' A(B-l) -j(B + l) 
-JA(B + \) -(5-1) 

Again, note that x = y for A = B = -\. Also note that for A = A opt and 

B = B opt 9 the matrices in Equations (6) and (7) are conjugate transpose pairs; in fact, it can 

be easily demonstrated that in this case they are unitary so that their product is the identity 
15 matrix and x = x. Thus, the compensator completely removes the modeled PMD for 
A = A opt andB = B Qpt . 

The difference between the ideal input signal vector x and its recovered 
estimate x forms an error signal vector 

c(z) = x(z)-x(z) (8) 
20 which should be small. In other words, the error signal vector, e(z), (having two values, 
one for each principle polarization mode) characterizes the difference between the signal 
that was transmitted, x(z), (which is known) and the response, x(z), of the PMD 
compensator. Here, it is assumed that compensation is performed over a discrete set of 
AT representative frequencies z k ,k = 1,..., AT, and form the weighted cost function: 

25 J = f J [^Hz k )\ 1 +X 1 \e 1 {z k )\ 1 ]. (9) 
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Equation (9) looks at the two components of the error signal vector, e(z), as functions of 
frequency (Zk). Generally, Equation (9) determines a weighted sum of the squared error, 
that is summed over K frequencies. 

As discussed hereinafter, the present invention provides adaptive 
5 algorithms for adjusting A and B to minimize this cost function, J. The complex 
(amplitude and phase) measurements of the optical PMD compensator output x for each 
frequency z k are provided by the detector 310 of FIG. 3. Generally, such measurements 

can be obtained using a tunable narrowband optical filter and various other optical 
components to render information from energy detector measurements. For a more 
10 detailed discussion of the detector 310 for measuring the complex (amplitude and phase) 
measurements of the optical PMD compensator output x for each frequency z k , see United 

States Patent Application Serial Number 10/180,842, entitled "Apparatus and Method for 
Measurement and Adaptive Control of Polarization Mode Dispersion in Optical Fiber 
Transmission Systems," incorporated by reference herein. 

1 5 Adaptive Algorithms 

According to another aspect of the invention, two exemplary algorithms are 
presented for adapting the coefficients of a two-channel adaptive filter structure involving 
two all-pass filters and two directional couplers. Both exemplary adaptive algorithms 
minimize the cost function of equation (9). The first adaptive algorithm is a stochastic 

20 gradient algorithm, also referred to as the least mean square algorithm, and sequentially 
reduces the value of J by the method of steepest descent. For the simplified model of 
FIGS. 5 and 6, the effects of random signals or noise are not considered, so the LMS 
algorithm in this case operates on deterministic signals (thus, the LMS algorithm is not 
really forming a stochastic gradient). 

25 As is well known, the LMS algorithm has the disadvantage of slow 

convergence for some applications in which the effect of the adjustable coefficients is 
closely coupled. As shown hereinafter, such behavior results for the all-pass PMD 
compensation context of the present invention. Therefore, a Newton algorithm is also 
derived for this application that makes use of second derivatives to accelerate 

30 convergence. For both algorithms, it will be necessary to calculate derivatives of the error 
signals e, and in (9). 
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LMS Algorithm 

The LMS algorithm adapts the (complex) coefficients of the PMD 
compensator all-pass filters employed by equations (3) and (5) as follows: 

a p (n+l)=a p (n)- M -^- (10a) 
p 

b q (n + \)=b q (n)- M ^- dOb) 
3/ 

where ju is the step size and — denotes the complex derivative of J with respect to the 

dc 

complex variable c, defined as: 

3/3.9 /t t x 

— = + / , (11) 

dc dVXc) 33(c) 

where 93(c) and 3(c) are, respectively, the real and imaginary parts of c=$%c)+f3(c). 

As previously indicated, these updates are deterministic for the simplified 
model. However, in the case of random signals, this also provides instantaneous gradient 
updates which are subsequently smoothed by the recursive nature of the algorithm, as 
controlled by the selection of //. In general, the selection of step size value must take into 
account the competing objectives of rapid convergence/tracking, and maintaining stability 
(and smoothing in the case of random signals). 

Differentiating equation (9) with respect to a p , it can be shown that: 

AT K 

~2j?LG p {z k W(z k ) (12a) 



^-=2jyiG p (z k m(z k ), (12b) 

where a p and d p are, respectively, the real and imaginary parts of a p =a r p +jd p , 



and 



( 1 ^ 



\ z ~ a pj 



(13) 



V?(z)-43Mz)Wz)-lto(*K(z)} (14) 
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10 



is a real valued function, independent of the index p. Combining equations (12a) and 
(12b), the complex derivative can be more compactly written as 



p 



(15) 



The complex derivative can be alternatively calculated by considering a p 

and d p to be independent variables and differentiating with respect to d p . However, the 

derivatives are separately derived with respect to the real and imaginary parts of a p 

because they will be needed in the next section to calculate the second derivatives. 
Likewise, for B, the following is obtained: 



k=\ 

K 



where 



f 1 ^ 



z-b. 



i J 



and 



15 and 



F 2 (z) =A&B(z)lAz)K (z) -jy 2 (z)ti(z)} 



A composite coefficient vector is defined as: 



w = 



(16a) 
(16b) 

(17) 
(18) 
(19) 
(20) 



where a = [a, a 2 ... a p ] r and b = [b,b 2 ... b Q ] T , then equation (10) can be compactly 
20 written as: 

w(« + l) = w(«)-//V(y), (21) 

where 
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V(J) 



lda T db T ] 



(22) 



is the (P + Q) x 1 complex gradient of 7 with respect to w, and 



dJ 
da 7 " 

dJ 

db T 



dJ dJ 

da x da 2 

dJ dJ 

db, db 2 



8J 



dJ 
db n 



(23a) 



(23b) 



It has been observed that the LMS algorithm demonstrates slow 
convergence, which can be solved using the Newton algorithm to accelerate convergence, 
discussed hereinafter. 

Newton Algorithm 

Newton's method involves multiplying the gradient by the inverse matrix of 
second derivatives, or Hessian. It has been found that it is not generally possible to apply 
this to the complex LMS algorithm defined by equation (21), whereby the Hessian is a 
complex matrix, even in the simplest case with a single complex coefficient. Therefore, 
the real and imaginary components of the weight update, as in equations (12) and (16), 
must be dealt with separately. 

Taking second derivatives of equation (9) with respect to the real and 
imaginary parts of a p and b p , it can be shown that: 

d 7irT = 2 Z h G « (** i (** ) + (** )]*; ( z * } ( 24a > 



da r q da p 



d 2 J 
da\da p 

d 2 J 



k=i 

K 



= 2^ {^[G q (z k )]3[G;(z 4 )]G, , (z k ) + m[G 2 p (z 4 )]F t (** } (24b) 



K 



'jfr = 2XhG,(z t )MG p (z t )]G 11 (z t )-3[G;(z t )]^(z i )^} (24c) 



da' q da p 



d 2 J 



*=1 
K 



^- = 2±{z[H q (z k mH p (z k )]G 22 (z k ) + ZlHfaW^sJ (25a) 



db r q db p 



d 2 j 



k=] 
K 



— = 2£ {*[//, {z k )]3[// p (z k )]G 22 (z k ) + 9t[H 2 p (z k )]/< (z k )5 pq } (25b) 

q p k-l 



Q P 
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^- = 2|;{5K[//,(zJM// /> (z il )]G 22 (z i )- ^H 2 p (z k W t (z k )S„} (25c) 

q p k-\ 

d 2 j d 2 j , , _ , , 



s*;a a ; 


a^;a6; 




aV 




aa;a&; 




av 










aV 









*=1 
K 

k=\ 
K 

k=i 

K 



(26a) 
(26b) 
(26c) 
(26d) 



where G^and H q arc defined, respectively, in (13) and (17), <? M =lfor p — q and 0 
otherwise, and 

G, , (z) ^ A,<R{2/l(z)[fi(z) - \]y x (z)<(z) + |[£(z) - 1]* (z)| 2 } 

+ A, 3{2A(z)[B(z) + llv, (z)e' (z) + y|[£(z) + l]y, (z)| 2 } (27a) 
G 22 (z)^^5R{25(z)M(z)^(z)- 7 > 2 (z)K(z) + |/i(z)^(z)-7> 2 (z)| 2 } 

+ ^3{25(z)[^)3' 1 (z)-7> 2 (^)K(z) + 7^(z)3; I (z)- y > 2 (z)| 2 } (27b) 
G 12 (z) = A,«{2^(z)5(z) 7l (z)e,*(z) + A * (z)2?(z)[5 * (z) - l][^(z)^,(z) - jfc,(z)K(z)} 
+ A,Z{2A(z)B(z)y } (z)e 2 (z) + jA * (z)B(z)[B * (z) + \}[A{z)y x (z) - y> 2 (z)]y t (z)} (27c) 
are real functions. 

It is noted that as e, — »0 and e 2 -*0, G,, and G 22 are non-negative, and 
F t — > 0 and F 2 — » 0 . Therefore, as the error goes to zero, equations (24) and (25) show 
that the diagonal terms 

d 2 J d 2 J d 2 J d 2 J 



da r p da r p da p da' p db r q 8b r q db'db' 



(28) 



are all non-negative, thus establishing a quadratic minimum. 

The P x P Hessian sub-matrix is now assembled from equation (24a) as 
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d 2 J 



da r da r 



d 2 J 


d 2 J 


d 2 j 


da r x da[ T 


da[da r 2 


da[da' p T 


d 2 J 


d 2 J 


d 2 J 


da r 2 da[ T 


da r 2 daf 


da r 2 da r p T 


d 2 J 


d 2 j 


d 2 J 


da r P da[ T 


da r P da r 2 T 


da r P da p T 



(29) 



Likewise, 



av 



aa'aa' 

2P x 1 coefficient vector as: 



•and 



d 2 J 



d 2 J 



da'da 1 da'da r 



Y are formed. Then, defining the 



a = 



where a r 


-K « 2 r ... 


and 




matrix: 




r d 2 j 


d 2 J 




d 2 J 


da r da' T 


da'8a iT 




dada T 


d 2 J 


3 2 J 






.da'da^ 


da'da' 7 



(30) 



a '„]> we form the 2Px2P Hessian 



(31) 



Similarly, from equations (25) and (26), the 2Qx2Q matrix is obtained as 



follows: 



10 



d 2 J 
dbdb T 



d 2 J 


d 2 j 


abw r 


ab r ab' r 


b 2 j 


d 2 j 


ab'ab rr 





(32) 



and the 2Q x 2P matrix is expressed as follows: 



8 2 J 
dbda r 







d 2 j 


d 2 j 


d 2 J ' 


T 


db'da rT 


8b'da iT 


dadb T _ 




d 2 J 








_db i da' T 


dWda iT 



(33) 



where 



b = 



b r 
b' 



(34) 
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The above results are consolidated and the Newton algorithm is then 
formulated. First, a composite real coefficient vector is defined as follows: 



w = [a a b bj = 



(35) 



and the composite (2P + 2Q)x(2P + 2Q) Hessian matrix is formed as follows: 

d 2 J d 2 J 



H = 



d 2 J 

<3w3w 7 



dada T 

d 2 j 



dadb r 

d 2 j 



(36) 



_dbda T dbdb T 

Similarly, using equations (12) and (16), the real gradient of J can be 
assembled with respect to the composite real coefficient vector (35) as follows: 

dJ dJ DJ 



dJ 

da iT 



alTjzl *r. (37) 



da rT 5a ir db rT 
Finally, with the above definitions, the Newton algorithm is written as 
10 w(w + 1) = w(#i) - juH~ l V(J) . (38) 

Generally, it has been observed that with the Newton algorithm, the error 
converges more quickly and steadily to zero, relative to the LMS algorithm which takes a 
longer time to converge completely. Even for 10% initialization with the Newton 
algorithm, convergence to zero was rapidly attained, although there may be some potential 
1 5 instability at some intermediate point. 

In one variation, the capability of the Newton algorithm is extended by first 
running the LMS algorithm for a small number of samples and then using the coefficient 
values to initialize the Newton algorithm. It is noted that both the LMS and Newton 
algorithms are susceptible to getting stuck in a local minimum. The theoretical basis for 
20 this phenomenon is related to the nature of the MSE gradient, which for the all-pass 
structure turns out to have components that only differ by a function that depends on the 
coefficient value. Therefore, if any two coefficients ever reach the same value they will 
remain locked together, thereby leading to the local minimum. One way around this 
problem is to compute multiple solutions starting with a suitable number of initial guesses 
25 spaced out over the feasible parameter space. 

As is known in the art, the methods and apparatus discussed herein may be 
distributed as an article of manufacture that itself comprises a computer readable medium 
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having computer readable code means embodied thereon. The computer readable program 
code means is operable, in conjunction with a computer system, to carry out all or some of 
the steps to perform the methods or create the apparatuses discussed herein. The computer 
readable medium may be a recordable medium (e.g., floppy disks, hard drives, compact 
disks such as DVD, or memory cards) or may be a transmission medium (e.g., a network 
comprising fiber-optics, the world-wide web, cables, or a wireless channel using time- 
division multiple access, code-division multiple access, or other radio-frequency channel). 
Any medium known or developed that can store information suitable for use with a 
computer system may be used. The computer readable code means is any mechanism for 
allowing a computer to read instructions and data, such as magnetic variations on a 
magnetic media or height variations on the surface of a compact disk, such as a DVD. 

It is to be understood that the embodiments and variations shown and 
described herein are merely illustrative of the principles of this invention and that various 
modifications may be implemented by those skilled in the art without departing from the 
scope and spirit of the invention. 



